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DETAILED ACTION 
Continued Examination Under 37 CFR 1.114 

1. A request for continued examination under 37 CFR 1.114, including the fee set 
forth in 37 CFR 1 .17(e), was filed in this application after final rejection. Since this 
application is eligible for continued examination under 37 CFR 1.114, and the fee set 
forth in 37 CFR 1 .17(e) has been timely paid, the finality of the previous Office action 
has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 
08/16/2007 has been entered. 

Claim Rejections - 35 USC § 103 

2. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

3. Claims 1-8, 10-30, and 36 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Zawilinski, U.S. Patent No. 5,676,138, in view of Hoffberg et al., U.S. 
Patent No. 6,400,996 in further view of Schmidt U.S. Patent No. 6,124,877. 

Regarding claim 1 , Zawilinski discloses a multimedia system comprising an 
emotional response analyzer. The system further comprises sensors for sensing 
the user's physical reactions (i.e., gaze, galvanic skin response, etc.) to various 
stimtrli, including commercials, wherein said stimuli are associated with 
recognizable emotional responses (i.e., surprise, anger, disgust, etc.). Moreover, 
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this information can be indexed and associated with whatever stimuli were being 
displayed at the time of said response. (Fig. 1 & 2; Abstract; Col. 1 , Ln. 6-47; Col. 
3, Ln. 39-Col. 4, Ln. 47; Col. 5, Ln. 34-43, 63-Col. 6, Ln. 26, 35-Col. 7, Ln. 5; Col. 
9, Ln. 50-64). But, Zawilinski fails to disclose a microphone picking up 
vocalizations made by the user. In an analogous art, however, Hoffberg discloses 
using biometric sensor data to create a dynamic user preference profile, wherein 
voice patterns of the user are detected and associated with a recognizable 
emotional response (i.e., moods; col. 116, 11. 3-14, col. 61, 11. 14-30, col. 65, 
U. 23-55; a microphone, or electro-acoustic transducer, is inherently present in 
where an acoustic signal (i.e., voice) is interpreted by an electrical device). Thus, 
it would have been obvious to one having ordinary skill in this art at the time of 
Applicant's invention to modify the at least one sensor of Zawilinski to include a 
microphone for picking up vocalizations, as taught by Hoffberg, to provide 
additional physiological data from which the user's preferences can be inferred. 
Still the combined teachings of Zawilinski and Hoffberg fail to disclose a 
reporting device that delivers results of the analysis of the sensor signal 
after a predetermined quantity of the television programming has been 
watched. In an analogous art, however, Schmidt discloses a System for 
monitoring and periodically reporting the viewing of television programs 
on a television receiver. (Col. 4 lines 61-67). Therefore it would have been 
obvious to one having ordinary skill in this art at the time of Applicant's 
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invention to modify the combined system of Zawilinski and Hoffberg, to 
report results of the analysis to researchers periodically. 

Claim 34 is encompassed within the rejection of Claim 1 . Thus, it is analyzed and 
rejected as previously discussed. 

As to claim 2, Zawilinski, Hoffberg, and Schmidt together disclose the claimed 
memory device (see Hoffberg, col. 113, U. 47-66). 

As to claim 3, Zawilinski further discloses the system comprises a plurality of 
sensors. (Col. 6, Ln. 3-26). 

As to claim 4, Zawilinski further discloses the claimed sensor-signal receiver 
(rigA, item 18; col. 6, 11. 27-35). 

As to claim 5, Zawilinski further teaches the system can analyze various 
physiological data including heart rate, galvanic skin response, etc. (i.e., 
aggregation of signals; see Zawilinski as applied to claim 1, above). 

As to claims 6-8, Zawilinski, Hoffberg , and Schmidt together disclose the use of 
a camera (video pattern recognition; Hoffberg, col. 116, 11.3-14) and video 
recorder recording images captured by the camera to an image library for 
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comparing to video images received from the camera (internal database; 
Hoffberg, col. 59, 11. 33-61). 

As to claim 10, Hoffberg, further discloses the system comprises an 
environmental sensor for sensing a change in the viewing environment, as 
claimed (col. 127, U. 5-40). 

Regarding claim 11, Zawilinski, Hoffberg, and Schmidt together disclose the 
claimed subject matter. (See the rejection of claim 1 ; Hoffberg discloses 
determining when a program segment is being received that corresponds to a 
p re . se | ec ted viewer response previously associated with a physical condition 
status (i.e., user's mood; col. 116, 11. 3-14, 33-49, col. 118, 11. 27-43).) 

As to claim 12, Zawfilinski, Hoffberg, and Schmidt together disclose monitoring a 
plurality of viewer physical conditions (Zawilinski, col. 6, 11.3-26). 

As to claim 13, Zawilinski further teaches the system can plot various stimuli 
changes over time (i.e., changes in condition relative to a baseline level). (Col. 9, 
Ln. 12-20). 



As to claim 14, Zawilinski, Hoffberg, and Schmidt together disclose the claimed 
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subject matter. In particular, Hoffberg discloses the physical condition is body 
temperature (col. 61, n. 13-30). 

As to claim 15, Zawilinski further teaches the system can analyzed the heart rate 
of a user, (see rejection of claim 1 ). 

As to claims 16 and 17, see the rejection of claims 6-8, above. 

As to Claim 18, Zawilinski, Hoffberg, and Schmidt together disclose the claimed 
subject matter. In particular, Hoffberg discloses determining characteristics of a 
displayed programming segment, associating a viewer response corresponding 
to a physical condition with a viewer preference level, and applying the 
preference level to enhance program selection (col. 100, 11. 2-51, col. 116, 11. 
33-49, col. 118, 11. 21-43). 



As to claim 19, Hoffberg further teaches providing a notification that specified 
future programming will contain at least one segment possessing the at least one 
distinguishing characteristic (col. 1 1 6, 1 . 50 - col. 117,1.5). 



As to claim 20, Hoffberg further teaches enhancing the program selection by 
inserting a segment possessing the distinguishing characteristic (col. 114, 11.12- 
23). 
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As to claims 21 and 22, Hoffberg further discloses the characteristic is derived 
from EPG information provided with the programming, including text (col. 150, U. 
38-54). 

As to claim 23, Hoffberg further discloses the claimed recorder for automatically 
recording the segment corresponding to a pre-selected viewer response (see 
portions of Hoffberg cited above). 

As to claim 24, Hoffberg further discloses extracting information related to the 
program segment that corresponds to pre-selected viewer response from the 
programming, and automatically displaying the information (col. 124, 11. 45-57). 

The limitations of claim 25 are encompassed within the portions of Zawilinski 
used to reject claim 1 . 

As to claim 26, Zawilinski further teaches the system can analyzed the galvanic 
skin response of a user. (Col. 6, Ln. 10-19). 

As to claims 27 and 28, Zawilinski further teaches the system is capable of 
measuring the gaze of the viewer and the electrical activity in muscles, which 
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relates to negative/positive facial expressions (i.e., visually observable 
response). (Col. 4, Ln. 13-31 and citations of Claim 1). 

As to Claim 29, Zawilinski further teaches the system can track the direction of 
the user's gaze. (Col. 9, Ln. 50-64). 

As to claim 30, Zawilinski further discloses the system is capable of analyzing the 
movement (i.e., changes in direction) of the viewers gaze and correlating the 
gaze direction with specific time periods (col. 9, 11. 50-64, col. 4, II. 42-47). 

As to claim 36, Zawilinski, Hoffberg, and Schmidt together disclose the method of 
claim 34, wherein an audible response is detected to infer the listener's mood. 
Official notice is taken that laughter is a well-known audible indication of mood. 
Thus, it would have been obvious to one of ordinary skill in the art at the time of 
the invention to modify the system of Zawilinski, Hoffberg, and Schmidt to include 
listener laughter as an audible response, thereby providing an addition indication 
of the listener's mood. 

4. Claims 31 and 32 rejected under 35 U.S.C. 103(a) as being unpatentable over 
Zawilinski in view of Hoffberg in view of Schmidt applied to claim 27 above, and further 
in view of Black et al. (US Patent No. 5,774,591). 
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Claim 31 recites the method of claim 27, wherein the visually observable 
response includes the furrowing of the viewer's brow. As discussed above, the 
combined systems of Zawilinski, Hoffberg, and Schmidt render obvious all 
limitations of Claim 27, and Zawilmski further teaches the system can analyzed 
the electrical impulses in the user's muscles (i.e., which can related to facial 
expressions), but fails to specifically recite analyzing eyebrow furrowing. 
However, within the same field of endeavor, Black et al disclose a similar system 
which analyzes changes in the viewer's eye brows. (Abstract; Col. 7, Ln. 1-45; 
Col. 26, Ln. 50-67; Col. 28, Ln. 28-62). Accordingly, it would have been obvious 
to one having ordinary skill in this art at the time of Applicant's invention to 
combine the systems of Zawilinski, Hoffberg, Schmidt, and Black in order to 
provide a system capable of analyzing facial features, thereby affecting the 
operation of a computer system. 

The limitations of claim 32 would be obvious variants of the limitations of claim 
31. Since Black et al allow for the analysis of user eyebrows, analyzing the depth 
of movement in said eyebrows would only be an obvious variant. Accordingly, it 
would have been obvious to one having ordinary skill in this art at the time of 
Applicant's invention to modify the combined systems of Zawilinski, Hoffberg, 
Schmidt and Black in order to provide a system which analyzes the depth of 
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furrows, thus allowing for an analysis of the level of a viewer facial response (i.e., 
level of anger, surprise, understanding, etc). 

5. Claims 33 and 38.are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Zawilinski, in view of Hoffberg, in view of Schmidt as applied to claims 1 1 and 34 
above, and further in view of Bentolila et al. (US Pat. Pub. No. 2003/0101449). 

Zawilinski, Hoffberg, and Schmidt together disclose the limitations of claims 1 1 
and 34, but fail to discuss the limitations of claims 33 and 38. However, within the 
same field of endeavor, Bentolila et al disclose a similar system which utilizes a 
Hidden Markov technique. (Par. [0021]). Accordingly, it would have been obvious 
to one having ordinary skill in this art at the time of Applicant's invention to 
combine the systems of Zawilinski, Hoffberg, and Schmidt and Bentolila in order 
to provide a system for user profile data prediction. 

6. Claim 37 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Zawilinski in view of Hoffberg, in view of Schmidt as discussed under claim 34, and 
further in view of Shinohara. (US Pat. Pub. No. 2003/0005431). 

Claim 37 recite the method of Claim 34, wherein the audibly observable 
response is the inflection (i.e., changes in pitch or tone) of a listener's 
vocalization, tending to indicate a question has been enunciated. As discussed 
above, the combined systems of Zawilinski, Hoffberg, and Schmidt render 
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obvious all limitations of Claim 34, but fail to specifically recite the limitations of 
Claim 37. However, within the same field of endeavor, Shinohara discloses a 
similar system which analyzes spectral data related to speech patterns, such as 
pitch, tone, frequency, etc., in order to identify a television viewer. (Par. [0026]). 
Accordingly, it would have been obvious to one having ordinary skill in this art at 
the time of Applicant's invention to combine the systems of Zawilinski, Hoffberg, 
Schmidt and Shinohara in order to provide a system which is capable of 
analyzing various tones, pitches, etc. of a given voice. 

Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Sahar A Baig whose telephone number is 571-270- 
3005. The examiner can normally be reached on 4/5/9. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Chris Kelley can be reached on 571-272-7331 . The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. - 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 
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